长期以来，Robotics一直是一个遍布复杂系统体系结构的领域，无论传统或基于学习的模块和联系都需要大量的人类专业知识和先验知识。受大型预训练语言模型的启发，这项工作引入了预先培训的通用表示范式，该范式可以作为给定机器人多个任务的起点。我们提出了感知性因果变压器（PACT），这是一种基于生成变压器的架构，旨在以自我监督的方式直接从机器人数据构建表示形式。通过对状态和行动的自动回归预测，我们的模型隐含地编码了特定机器人的动态和行为。我们的实验评估重点是移动药物的域，我们表明该机器人特定的表示可以作为单个起点，以实现不同的任务，例如安全导航，定位和映射。我们评估了两个形式：使用激光雷达传感器作为感知输入（MUSHR）的轮式机器人，以及使用第一人称RGB图像（栖息地）的模拟药物。我们表明，与训练单个模型的同时训练单个模型相比，对所有任务的单个模型进行训练，并且与独立培训单独的大型模型相当的性能，对每个任务的单个模型进行了可比的训练，则在较大的审计模型上进行了固定小型任务特异性网络，从而使性能明显提高。通过跨任务共享共同的优质表示，我们可以降低整体模型容量并加快此类系统的实时部署。

由于其物理能力，模拟的类人动物是一个吸引人的研究领域。尽管如此，他们也在控制方面具有挑战性，因为政策必须推动不稳定，不连续和高维物理系统。一种经过广泛研究的方法是利用运动捕获（MOCAP）数据来教授类人动物的低水平技能（例如，站立，步行和跑步），然后可以重新使用以综合高级行为。但是，即使使用MOCAP数据，控制模拟的类人动物仍然非常困难，因为MOCAP数据仅提供运动学信息。寻找物理控制输入以实现所示动作需要计算密集型方法，例如增强学习。因此，尽管有公开可用的MOCAP数据，但其效用仍限于具有大规模计算的机构。在这项工作中，我们通过训练和释放高质量的代理，可以大大降低有关该主题的生产研究的障碍，这些代理可以在基于DM_Control物理学的环境中跟踪三个小时的MOCAP数据以上的MOCAP数据。我们释放Mocapact（动作动作捕获），这些专家代理的数据集及其推出，其中包含本体感受观察和动作。我们通过使用它来训练单个层次结构策略来证明MOCAPACT的实用性，该策略能够跟踪DM_Control中的整个MOCAP数据集并显示学习学到的低级组件可以被重新使用以有效地学习下游高级任务。最后，我们使用MoCapact训练自动回旋GPT模型，并表明它可以控制模拟的类人动物以在运动提示下执行自然运动完成。结果和指向代码和数据集的链接的视频可在https://microsoft.github.io/mocapact上获得。

我们为处理顺序决策和外在不确定性的应用程序开发了增强学习（RL）框架，例如资源分配和库存管理。在这些应用中，不确定性仅由于未来需求等外源变量所致。一种流行的方法是使用历史数据预测外源变量，然后对预测进行计划。但是，这种间接方法需要对外源过程进行高保真模型，以确保良好的下游决策，当外源性过程复杂时，这可能是不切实际的。在这项工作中，我们提出了一种基于事后观察学习的替代方法，该方法避开了对外源过程进行建模的建模。我们的主要见解是，与Sim2real RL不同，我们可以在历史数据中重新审视过去的决定，并在这些应用程序中对其他动作产生反事实后果。我们的框架将事后最佳的行动用作政策培训信号，并在决策绩效方面具有强大的理论保证。我们使用框架开发了一种算法，以分配计算资源，以用于现实世界中的Microsoft Azure工作负载。结果表明，我们的方法比域特异性的启发式方法和SIM2REAL RL基准学习更好的政策。

A Digital Twin (DT) is a simulation of a physical system that provides information to make decisions that add economic, social or commercial value. The behaviour of a physical system changes over time, a DT must therefore be continually updated with data from the physical systems to reflect its changing behaviour. For resource-constrained systems, updating a DT is non-trivial because of challenges such as on-board learning and the off-board data transfer. This paper presents a framework for updating data-driven DTs of resource-constrained systems geared towards system health monitoring. The proposed solution consists of: (1) an on-board system running a light-weight DT allowing the prioritisation and parsimonious transfer of data generated by the physical system; and (2) off-board robust updating of the DT and detection of anomalous behaviours. Two case studies are considered using a production gas turbine engine system to demonstrate the digital representation accuracy for real-world, time-varying physical systems.

Training agents via off-policy deep reinforcement learning (RL) requires a large memory, named replay memory, that stores past experiences used for learning. These experiences are sampled, uniformly or non-uniformly, to create the batches used for training. When calculating the loss function, off-policy algorithms assume that all samples are of the same importance. In this paper, we hypothesize that training can be enhanced by assigning different importance for each experience based on their temporal-difference (TD) error directly in the training objective. We propose a novel method that introduces a weighting factor for each experience when calculating the loss function at the learning stage. In addition to improving convergence speed when used with uniform sampling, the method can be combined with prioritization methods for non-uniform sampling. Combining the proposed method with prioritization methods improves sampling efficiency while increasing the performance of TD-based off-policy RL algorithms. The effectiveness of the proposed method is demonstrated by experiments in six environments of the OpenAI Gym suite. The experimental results demonstrate that the proposed method achieves a 33%~76% reduction of convergence speed in three environments and an 11% increase in returns and a 3%~10% increase in success rate for other three environments.

We describe a Physics-Informed Neural Network (PINN) that simulates the flow induced by the astronomical tide in a synthetic port channel, with dimensions based on the Santos - S\~ao Vicente - Bertioga Estuarine System. PINN models aim to combine the knowledge of physical systems and data-driven machine learning models. This is done by training a neural network to minimize the residuals of the governing equations in sample points. In this work, our flow is governed by the Navier-Stokes equations with some approximations. There are two main novelties in this paper. First, we design our model to assume that the flow is periodic in time, which is not feasible in conventional simulation methods. Second, we evaluate the benefit of resampling the function evaluation points during training, which has a near zero computational cost and has been verified to improve the final model, especially for small batch sizes. Finally, we discuss some limitations of the approximations used in the Navier-Stokes equations regarding the modeling of turbulence and how it interacts with PINNs.

Most TextVQA approaches focus on the integration of objects, scene texts and question words by a simple transformer encoder. But this fails to capture the semantic relations between different modalities. The paper proposes a Scene Graph based co-Attention Network (SceneGATE) for TextVQA, which reveals the semantic relations among the objects, Optical Character Recognition (OCR) tokens and the question words. It is achieved by a TextVQA-based scene graph that discovers the underlying semantics of an image. We created a guided-attention module to capture the intra-modal interplay between the language and the vision as a guidance for inter-modal interactions. To make explicit teaching of the relations between the two modalities, we proposed and integrated two attention modules, namely a scene graph-based semantic relation-aware attention and a positional relation-aware attention. We conducted extensive experiments on two benchmark datasets, Text-VQA and ST-VQA. It is shown that our SceneGATE method outperformed existing ones because of the scene graph and its attention modules.

Existing analyses of neural network training often operate under the unrealistic assumption of an extremely small learning rate. This lies in stark contrast to practical wisdom and empirical studies, such as the work of J. Cohen et al. (ICLR 2021), which exhibit startling new phenomena (the "edge of stability" or "unstable convergence") and potential benefits for generalization in the large learning rate regime. Despite a flurry of recent works on this topic, however, the latter effect is still poorly understood. In this paper, we take a step towards understanding genuinely non-convex training dynamics with large learning rates by performing a detailed analysis of gradient descent for simplified models of two-layer neural networks. For these models, we provably establish the edge of stability phenomenon and discover a sharp phase transition for the step size below which the neural network fails to learn "threshold-like" neurons (i.e., neurons with a non-zero first-layer bias). This elucidates one possible mechanism by which the edge of stability can in fact lead to better generalization, as threshold neurons are basic building blocks with useful inductive bias for many tasks.

Generative Adversarial Networks (GANs) were introduced by Goodfellow in 2014, and since then have become popular for constructing generative artificial intelligence models. However, the drawbacks of such networks are numerous, like their longer training times, their sensitivity to hyperparameter tuning, several types of loss and optimization functions and other difficulties like mode collapse. Current applications of GANs include generating photo-realistic human faces, animals and objects. However, I wanted to explore the artistic ability of GANs in more detail, by using existing models and learning from them. This dissertation covers the basics of neural networks and works its way up to the particular aspects of GANs, together with experimentation and modification of existing available models, from least complex to most. The intention is to see if state of the art GANs (specifically StyleGAN2) can generate album art covers and if it is possible to tailor them by genre. This was attempted by first familiarizing myself with 3 existing GANs architectures, including the state of the art StyleGAN2. The StyleGAN2 code was used to train a model with a dataset containing 80K album cover images, then used to style images by picking curated images and mixing their styles.

We introduce M-VADER: a diffusion model (DM) for image generation where the output can be specified using arbitrary combinations of images and text. We show how M-VADER enables the generation of images specified using combinations of image and text, and combinations of multiple images. Previously, a number of successful DM image generation algorithms have been introduced that make it possible to specify the output image using a text prompt. Inspired by the success of those models, and led by the notion that language was already developed to describe the elements of visual contexts that humans find most important, we introduce an embedding model closely related to a vision-language model. Specifically, we introduce the embedding model S-MAGMA: a 13 billion parameter multimodal decoder combining components from an autoregressive vision-language model MAGMA and biases finetuned for semantic search.